Simplified molecular diagnosis of visceral leishmaniasis: Laboratory evaluation of miniature direct-on-blood PCR nucleic acid lateral flow immunoassay

Background Diagnosis of visceral leishmaniasis (VL) in resource-limited endemic regions is currently based on serological testing with rK39 immunochromatographic tests (ICTs). However, rK39 ICT frequently has suboptimal diagnostic accuracy. Furthermore, treatment monitoring and detection of VL relapses is reliant on insensitive and highly invasive tissue aspirate microscopy. Miniature direct-on-blood PCR nucleic acid lateral flow immunoassay (mini-dbPCR-NALFIA) is an innovative and user-friendly molecular tool which does not require DNA extraction and uses a lateral flow strip for result read-out. This assay could be an interesting candidate for more reliable VL diagnosis and safer test of cure at the point of care. Methodology/Principle findings The performance of mini-dbPCR-NALFIA for diagnosis of VL in blood was assessed in a laboratory evaluation and compared with the accuracy of rK39 ICTs Kalazar Detect in Spain and IT LEISH in East Africa. Limit of detection of mini-dbPCR-NALFIA was 650 and 500 parasites per mL of blood for Leishmania donovani and Leishmania infantum, respectively. In 146 blood samples from VL-suspected patients from Spain, mini-dbPCR-NALFIA had a sensitivity of 95.8% and specificity 97.2%, while Kalazar Detect had a sensitivity of 71.2% and specificity of 94.5%, compared to a nested PCR reference. For a sample set from 58 VL patients, 10 malaria patients and 68 healthy controls from Ethiopia and Kenya, mini-dbPCR-NALFIA had a pooled sensitivity of 87.9% and pooled specificity of 100% using quantitative PCR as reference standard. IT LEISH sensitivity and specificity in the East African samples were 87.9% and 97.4%, respectively. Conclusions/Significance Mini-dbPCR-NALFIA is a promising tool for simplified molecular diagnosis of VL and follow-up of treated patients in blood samples. Future studies should evaluate its use in endemic, resource-limited settings, where mini-dbPCR-NALFIA may provide an accurate and versatile alternative to rK39 ICTs and aspirate microscopy.


INTRODUCTION General remarks:
-Comment reviewer: The background section is rather long and contains information that is not necessarily important to understand the content of the manuscript.For example, the authors explain in depth the impact of the disease in East-Africa, then again in the Mediterranean and then focus on outbreaks in southern Europe (nothing about Indian subcontinent?).Please try to reduce this part.Most important is that there is two main causative species in different parts of the world and that due to the elimination initiative in the Indian subcontinent majority of cases are now reported from East-Africa, where available diagnostics have major limitations.

Response authors:
The authors thank the reviewer for this suggestion to improve the introduction.This section has been revised and shortened.-Comment reviewer: Authors mention in Line 84 that the gold standard for VL diagnosis is demonstration of Leishmania parasites in tissue aspirates through microscopy, however, this is only the case in resource-limited countries.I also do not completely agree that this is the gold standard, as it is only the last tool that will be employed if the other available methods in the diagnostic algorithm fail to detect Leishmania infection.Please revise.
Response authors: see below.-Comment reviewer: Related to the comments above, the section about tissue aspiration is too lengthy and needs to be reduced.In line 89, The authors state that tissue microscopy is unsuited for rural settings, however, in general because of the invasiveness and low sensitivity of the method it is not a preferable approach.
Response authors: please see below.-Comment reviewer: Overall I think the background would benefit from a shift in the structure, following the diagnostic algorithm.

Response authors:
The authors thank the reviewer for this suggestion to improve the background on currently used diagnostics.The second paragraph has been rewritten and shortened, first discussing rK39 as primary diagnostic, then mentioning its limitations and ending with the lack of a feasible ToC, as neither rK39 nor microscopy are suited.-Comment reviewer: An important advantage of the use of molecular tools is that it can be employed on less-invasive samples like blood.
Response authors: This advantage has been added (line 132).-Comment reviewer: The authors mention that molecular methods are not employed yet with the focus on East-Africa, however, to my knowledge it is also not routinely employed in the Americas or Indian subcontinent.The authors should either broaden the scope or highlight that the rK39 RDT diagnostic accuracy is suboptimal in East-Africa and therefore diagnosis often relies on tissue aspiration which does not allow for decentralization of diagnosis and care.

Response authors:
The scope of this sentence has been broadened as suggested (line 137) -Comment reviewer: The authors could still highlight that in Sudan, it was recommended to incorporate LAMP in the diagnostic algorithm for VL, before tissue microscopy would be done.However, major limitations to do so are indeed the mentioned disadvantages, but I think also still the cost of a LAMP assay.

Response authors:
The limitation of the costs has been added to this paragraph (line 150-151).To avoid that the paragraph becomes too lengthy, the example of Sudan has not been added.Response authors: During submission of the manuscript, the original submitted word document was erroneously altered by the submission portal software: part of the method section was removed and inserted later in the manuscript.Unfortunately, the authors only noticed this when the manuscript had already been send out for review.The revised document does not contain this error, and the reaction conditions are described in lines 213 -215 -Comment reviewer: Can the authors explain why they did not go lower than 10 par/mL for the LoD?The LoD of kDNA is known to be much lower than that.
Response authors: dbPCR is performed directly on 2.5 µL of blood, without DNA concentration step.This means that 10 0 p/mL equals 0.0025 p/reaction, a very low density where the chance of stochastic variation between tests becomes significant.10 0 p/mL had been tested in an earlier phase and consistently gave negative results with NALFIA.This concentration was therefore not included in the 20 repeats for the determination of the LoD.-Comment reviewer: Why are the results of the rK39 RDT already mentioned in the methods if this was a study procedure and so a result?Response authors: The method section on the sens/spec evaluation has been rewritten to clarify that all samples used in this study had already been tested with rK39 ICT previously, at the time of collection.Hence, rK39 testing was not a study procedure, these results were already available and were used for the selection of Kenyan and Ethiopian samples for this study.-Comment reviewer: Perhaps it would help to have a flow chart or table in the methods to clarify the sample sets used for testing: where samples came from (especially in East-Africa: it is strange it comes up in the last sentence only and is still unclear), what was considered a VL-suspected patient, whether it was a prospective collection on secondary use of samples (in case of the latter, how were samples selected considering that authors knew which ones were -strongly -positive and selected exactly half negatives/positives), how they were tested previously/for the study etc.Going through the text, it is not clearly described and numbers do not seem to add up but that is perhaps due to the fact that the way it is described is a bit confusing.It would be useful if the authors considerably revise this section and add a figure or table to guide it.

Response authors:
We thank the reviewer for pointing out that this description was not clear and for the suggestion of a supporting figure as well.The section on the sensitivity/specificity evaluation has been rewritten to clarify where the samples originated from, how they were selected and which procedures were done previously and which specifically for this study.Two supplementary figures (S1 and S2) were made to visualize this.-Comment reviewer: Can the authors clarify the cut-off of 36 they used for positivity of the kDNA PCR or refer to a paper that describes this?
Response authors: This cut-off has been based on in-house (unpublished) validation of the kDNA qPCR and renders the most accurate results in terms of sensitivity and specificity.This was added to the manuscript (line 362) --------------------

Results
Reviewer #1: Comment reviewer: [reviewer made a comment about discrepant numbers in Table 2.] Response authors: To comment on the discrepant numbers in Table 2: the authors thank the reviewer for noticing this.This was because 2 samples from Spain had an invalid NALFIA result and were excluded from the NALFIA comparison with LnPCR, but not for the rK39 ICT -LnPCR comparison.This has now been indicated in table 2.
Comment reviewer: Table 3: Could it be that scoring of qPCR thresholds may have included false positives.Were denaturation curves run to verify qPCR products?Alternatively, could the mismatches in the strip assay have limited detection in the East African context?
Response authors: No denaturation curves were run to verify the specificity of the PCR products, meaning that we cannot exclude the occurrence of some false-positive results in qPCR.This was added to the discussion (line 594-595).However, we consider the cut-off of 36 relatively conservative, and based on in-house validation of the kDNA qPCR at AMC, false-positive signals with a Cq <36 are deemed unlikely.The reliability of the qPCR results is also supported by the sensitivity estimates for rK39 ICT in East Africa which match with literature reports (see revised discussion).
Regarding the mismatch hypothesis, please refer to our comment above.

RESULTS
-Comment reviewer: It is not entirely clear to me how the authors calculated with this much depth the LoD of the mini-dbPCR-NALFIA that the difference between 500 and 650 par/mL can be detected.

Response authors:
To get a first idea of the LoD, only 10-fold dilutions were tested.If all 20 tests were positive for a given concentration, but the majority of tests for one concentration lower was negative, an intermediate concentration between these two was tested to estimate the LoD with more precision.For example, for L. donovani all 20 tests were positive for 1000, but for 100 and 500 p/mL, 19/20 tests were negative.Therefore, intermediate concentrations of 750 and 650 p/mL were tested to determine the LoD in more depth.To clarify this approach, a sentence was added to the LoD methods (lines 242-243) -Comment reviewer: Line 385: were these samples that were mini-dbPCR-NALFIA positive also positive by kDNA PCR?Response authors: As LnPCR was the reference test for the samples from Spain, kDNA qPCR was only performed on the LnPCR-positive samples to quantify their parasite densities, and on a small number of randomly selected LnPCR-negative samples (as negative controls).This subset included one of the two samples that was NALFIA positive: it was negative in the kDNA qPCR, suggesting that it was truly negative and the positive NALFIA result may have been due to contamination.-Comment reviewer: To show the agreement between the different tests, instead of only using values in the text, authors could perhaps add a (supplementary) VENN diagram to show the overlapping results in the three sample sets.
Response authors: Two Venn diagrams were created to visualise the results for Spain and East Africa (supplementary figures S3 and S4).

FIGURES AND TABLES -Comment reviewer:
The authors should ensure that the figures and tables are presented in the text at the right time and should link to it in their text.

Response authors:
The authors have checked the figure and table references in the text and tables were moved and placed directly after the paragraph where they were first mentioned.-Comment reviewer: The authors indicate in figure 1 that a band for kDNA but not for GAPDH is still considered positive.Is that a likely possibility?Could it be contamination is this case?Response authors: In principle, the GAPDH line should always be positive, since human GAPDH DNA is present in any human blood sample that is tested.However, due to the high copy number of kDNA, it has been observed that amplification of the GAPDH target can be inhibited by the amplification of the kDNA target in case of high parasite input.
Contamination could be a possibility as well, this cannot be distinguished from the scenario explained above.-Comment reviewer: There is no reference in the text to Table 2.

Conclusions
Reviewer #1: Comment reviewer: Line 365: "The sensitivity of mini-dbPCR-NALFIA in East African samples was found to be slightly lower" But using a different reference test.Could there be limitation of reference tests?
Response authors: The authors agree that the difference may have been affected by the difference in reference test, as is mentioned in the same paragraph.The limitation of using different references has been mentioned in the second last paragraph of the discussion, and the absence of a melting curve analysis has been added there.
Reviewer #2: -Comment reviewer: As throughout the rest of the text, also here the authors can be more comprehensive.The first Alinea of the discussion is repetitive from the introduction and repeats again what the overall aim of the study was.Methods used are explained again and results are repeated.Authors should focus more on how results should be interpreted and placed in a broader context.

Response authors:
The authors acknowledge that the original discussion was too lengthy and have rewritten it to make it more succinct.-Comment reviewer: Authors mention that further optimization could be done to improve the LoD but do not mention what can be done to indicate that indeed there is potential that it could be improved.I was a bit surprised to see that kDNA, which normally has a very low LoD in PCR, resulted in a high LoD.Is this most likely due to the direct PCR on blood without extraction or due to the read-out with the lateral flow?Can the authors reflect on this?Also the word "could" should probably be "should", because with the current diagnostic accuracy I do not see how otherwise it would ever be implemented in routine practice in the field.

Response authors:
The authors agree that further optimization will indeed be necessary to make dbPCR-NALFIA suitable for routine practice.During the development of the assay, high specificity was initially prioritized over a low LoD (due to the burden and toxicity of antimonial treatment used in East Africa, false-positive results should be prevented).Suggestions for (PCR) optimization have been added in line 499-500.The NALFIA read-out may have less potential for optimization, as increasing the volume of PCR product loaded on the strip has so far not resulted in higher sensitivity in our experiments.-Comment reviewer: The authors state: "The lower sensitivity in East Africa may have resulted from parasite densities close to the LoD of mini-dbPCR-NALFIA, found in a number of samples and leading to a false-negative NALFIA result."These data are available from the qPCR right?Can the authors add these data to the results and make the statement stronger?
Response authors: The parasite densities of the false-negative NALFIA results are mentioned in the result section.-Comment reviewer: It is mentioned that the InBios RDT in Spain had a considerably lower sensitivity than the dbPCR-NALFIA, which was not the case in East-Africa (where IT-Leish is used).Could this be intrinsic to the diagnostic accuracy of the specific RDT brand?Can the authors put this finding in a broader context and compare the obtained sens/spec with the literature.Or is it possible that if the RDT was used on stored samples that its sensitivity decreased as it's normally employed as a POC test?To clarify this, authors should refer more to existing literature.
Response authors: Throughout the manuscript, the brand difference has been highlighted more, and it has been mentioned in the discussion to explain the difference in performance.
All rK39 ICTs for this study were performed at the time of sample collection, which has been more clearly explained in the revised methods.-Comment reviewer: In the section about the employment of the dbPCR-NALFIA in East-Africa the authors should focus their statement particularly on the test-of-cure and diagnosis of relapsing patients.Except if the sensitivity can greatly be increased, it will probably not be employed in EA.However, for example in northwest Ethiopia, there is a very high proportion of HIV-VL coinfected patients, who still frequently undergo an invesive TOC and are repeatedly relapsing.For such populations, its employment would mostly be beneficial.

Response authors:
We agree with this reflection, the section has been adapted.-Comment reviewer: It would be good if the authors can explain in the manuscript what the cost is of the current test, including instruments needed.In order for the test to be costbeneficial, also in Spain, it should not only be easy and have good diagnostic accuracy, but also be cheap, unless of course if it would only be used for few patients.This is an important operational challenge that is often not described, but critical information for decision makers.
Response authors: A paragraph on the costs of mini-dbPCR-NALFIA has been added (line 567-573) -Comment reviewer: The strengths and weaknesses of the study can be written more concise.
Response authors: This section has been shortened.-Comment reviewer: In the conclusion, rather than using the test for POC diagnosis in place of the rK39 RDT, I think more focus can be put on its use for test-of-cure and relapsing patients, and so to get rid of the highly invasive aspirates for patients.Furthermore, authors can be more careful with their statement about Spain as well if the costs are higher than the lateral flow assays, because then take-up of the dbPCR-NALFIA would probably still not be as a first-line diagnosis, but maybe when the rK39 RDT is negative and there is still high suspicion of VL.

Response authors:
The conclusion was rewritten to emphasize the potential of the assay as test of cure and for relapse diagnosis.-Comment reviewer: Another reflection, however, is that as a test-of-cure one would like to have a very sensitive test, because even blood parasite load does not mean that the patient is completely cleared.So even though the dbPCR-NALFIA is less invasive, its LoD is still not really optimal for TOC and so should be further improved indeed.

Response authors:
We agree with this remark and this has been mentioned in the discussion (line 497-499).As mentioned by the reviewer, mini-dbPCR-NALFIA would already be a significant improvement compared to microscopy, both in terms of sensitivity and sampling invasiveness.-Comment reviewer: The authors describe a couple of times in their text that the test under validation is very robust.However, it did not provide valid results for some of the samples, which is not discussed anymore in the rest of the manuscript, but which indicates that it is not completely robust.Can the authors further elaborate on this and maybe compare this with the validity of other tests employed in routine?Response authors: In total, only 2 of the 282 tested samples had an invalid test result, which may have resulted from an error during PCR preparation.This is indeed not 100% robust, but nevertheless very robust, also when considering that the test performed well in different laboratories and in the hands of different operators.Some adaptations have been made in the manuscript, including an added sentence in lines 582-583.No literature has been found on frequency of invalid rK39 ICT results (IT LEISH or Kalazar Detect) in routine practice, hence no comparison could be made here --------------------

-
Comment reviewer:In line 126, can the authors specify that one target is to check for DNA extraction efficiency and indicate which Leishmania marker is targeted?kDNA?If so, maybe highlight why kDNA was selected as target.The objective of the study has been reformulated.As this is the first study to evaluate the mini-dbPCR-NALFIA for Leishmania diagnosis, no previous diagnostic accuracy can be reported.The assay has been evaluated before for Plasmodium (see: PMC10024383) but since this concerns a different pathogen with different PCR targets, it is not representative of the accuracy that may be expected for the Leishmania assay.

Response authors: Specifications on the export of the East African samples to the Netherlands have been added. General comments -Comment reviewer: More information needs to be provided on the source of
the samples/study population.Was it secondary use of biobanked samples or fresh samples?If the former, did patients consent to the use of their samples for secondary research?What did the patient population look like?What was used as a definition of VL? Was any sample size calculation performed?Please elaborate futher on study population and sample collection/storage in a separate paragraph.Response authors: A separate section on the used sample sets has been added, addressing the points mentioned by the reviewer.The patient consent for sample use has been added to the ethical clearance section.Sample size calculations are described in lines 253 -258.-Commentreviewer: It seems like part of the methods section is missing after line 162.What were the reaction conditions used?